Dynamic response entry

ABSTRACT

Methods, systems, and devices for dynamic response entry are disclosed herein. In some embodiments, a dynamic response entry system can include a user device that can be a proctor device or a testee device. The testee device can display a list to a testee for a predetermined time period. After the passing of the predetermined time period, the displaying of the list to the testee can be terminated. The testee can provide response to one or several questions, which responses can be input into the proctor device. The input responses can be evaluated and categorized and displayed according to the evaluation and categorization.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/468,012, filed Mar. 23, 2017, and entitled “DYNAMIC RESPONSE ENTRY,”which is a continuation of U.S. patent application Ser. No. 14/284,281,filed May 21, 2014, and entitled “DYNAMIC RESPONSE ENTRY,” now U.S. Pat.No. 9,836,986, which claims the benefit of U.S. Provisional ApplicationNo. 61/825,787, filed on May 21, 2013, and entitled “DYNAMIC RESPONSEENTRY,” the entirety of each which are incorporated by reference herein.

BACKGROUND OF THE INVENTION

This disclosure relates in general to learning and can includetraditional classroom learning or on-line or computerized learning, andspecifically to testing that can be a part of or associated withlearning. Testing can be used to verify the degree to which a studenthas mastered a subject and/or developed one or several skills. Incertain tests, a proctor received answers from the student and creates arecord of these answers. Traditionally, limitations on the speed withwhich the record of the answers can be created has limited theeffectiveness of this type of test. Thus, better methods and systems aredesired to improve testing.

BRIEF SUMMARY OF THE INVENTION

One aspect of the present disclosure relates to a method of dynamicresponse entry. The method includes receiving a task from a batterydatabase, which battery database can include a plurality of tasksassociated with a test, receiving list data from a list database, whichlist data can include data indicating a plurality of items, andproviding a task to a user, which task can include temporarily providingthe list to the user and requesting the user to identify the items fromthe list. The method can include receiving an input, which inputindicates a user identified item, and identifying a group associatedwith the list. In some embodiments, identifying the group can includereceiving data from the list database indicating the group for at leastone of the items. The method can include determining an input status,which input status can indicate whether the input identifies an item inthe list. In some embodiments, determining whether the input identifiesan item in the list can include converting the input to a format anddata type corresponding to the format and data type of the list data,and determining if the input is identified in the list data. The methodcan include associating an identifier with input, which identifierindicates if the input is identified in the list data. In someembodiments, the identifier is associated with the input according to aBoolean function. In one such embodiment, a first value is assigned ifthe input identifies an item that is identified in the list data and asecond value is assigned if the input identifies and item that is not inthe list data. The method can include displaying the input and the inputstatus.

In some embodiments, determining the input status can includedetermining if the input is identified in the group associated with thelist. In some embodiments, the method can include associating a secondidentifier with input, which the second identifier indicates if theinput is in the group associated with the list. In some embodiments, thesecond identifier is associated with the input according to a Booleanfunction, and in some embodiments a first value is assigned if the inputidentifies an item that is identified in the group and a second value isassigned if the input identifies and item that is not in the group.

In some embodiments of the method, determining the input status furtherincludes determining if the input is a repeat input. In someembodiments, determining if the input is a repeat input can includeretrieving input data from a response database, which input dataidentifies a previously received input, comparing the input to the inputdata, and identifying the input as a repeat input if the input matchesinput data corresponding to a previously received input. In someembodiments, the method includes associating a third identifier withinput, which third identifier indicates if the input is a repeat input.In some embodiments, the third identifier is associated with the inputaccording to a Boolean function, and in one such embodiment a firstvalue is assigned if the input is a repeat input and a second value isassigned if the input identifies and item that is not a repeat input.

In some embodiments of the method, displaying the input and the inputstatus can include retrieving the identifier associated with the inputand displaying an input status according to the identifier associatedwith the input. In some embodiments, the identifier can indicate one ofwhether the input is included in a list or group and whether the inputis a repeat input.

One aspect of the present disclosure relates to a method of dynamicresponse entry. The method includes receiving a task from a batterydatabase, which battery database can include a plurality of tasksassociated with a test, receiving list data from a list database, whichlist data can include data indicating a plurality of items, andproviding a task a user, which task can include temporarily providingthe list to the user and requesting the user to identify the items fromthe list. The method can include providing an input region, receiving afirst input, which first input can be the electronic capture ofhandwriting indicating a user identified item in a first input area,receiving an indication of the completion of the providing of the firstinput, providing an additional input region, determining whether anadditional input is provided, which can include determining whether asignal indicative of an additional input is received, and storing thefirst input.

In some embodiments, the method can include displaying the first inputin an answer list, and in some embodiments, the method includesdetermining a status of the first input. In some embodiments, the statusof the first input indicates at least one of whether the input isincluded in a list or group and whether the input is a repeat input. Insome embodiments, the method includes determining the status of thefirst input. This determination can include retrieving categorizationdata from a database and comparing the categorization data to the firstinput. In some embodiments, the categorization data can include one oflist data, which list data can include data indicating a plurality ofitems, group data, which group data can identify a subset of theplurality of items in the list data, and input data, which input datacan identify a previously received input.

In some embodiments, the method includes providing a button region tothe user, which button region can include a plurality of input buttons.In some embodiments, providing the button region can include generatingthe plurality of input buttons. In some embodiments, each of theplurality of input buttons can be associated with an item identified inlist data. In some embodiments, the list data can include dataindicating a plurality of items.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is described in conjunction with the appendedfigures:

FIG. 1 is a block diagram illustrating one embodiment of a dynamicresponse entry system.

FIG. 2 is a block diagram illustrating one embodiment of a user devicefor use with dynamic response entry system.

FIG. 3 is a schematic illustration of one embodiment of a user interfacethat is part of the user device.

FIG. 4 is a flowchart illustrating one embodiment of a process fordynamic response entry.

FIG. 5 is a flowchart illustrating one embodiment of a process forreceiving an input.

FIG. 6 is a schematic illustration of an exemplary environment withwhich embodiments may be implemented is shown with a computer system.

FIG. 7 is a schematic illustration of one embodiment of aspecial-purpose computer system.

In the appended figures, similar components and/or features may have thesame reference label. Where the reference label is used in thespecification, the description is applicable to any one of the similarcomponents having the same reference label. Further, various componentsof the same type may be distinguished by following the reference labelby a dash and a second label that distinguishes among the similarcomponents. If only the first reference label is used in thespecification, the description is applicable to any one of the similarcomponents having the same first reference label irrespective of thesecond reference label.

DETAILED DESCRIPTION OF THE INVENTION

The ensuing description provides preferred exemplary embodiment(s) only,and is not intended to limit the scope, applicability or configurationof the disclosure. Rather, the ensuing description of the preferredexemplary embodiment(s) will provide those skilled in the art with anenabling description for implementing a preferred exemplary embodiment.It is understood that various changes may be made in the function andarrangement of elements without departing from the spirit and scope asset forth in the appended claims.

In one embodiment, the present disclosure provides a method for dynamicresponse entry including, temporarily providing a list of words to auser, receiving an input from the user allegedly identifying one of thewords of the list, and characterizing the user provided input. In someembodiments, this characterization can include identifying whether theuser input accurately identifies one of the words of the list,identifying whether the user input repeats an already provided input,and/or determining whether the user input is in a category associatedwith one of the words in the list of words.

In one embodiment, the present disclosure provides a method for dynamicresponse entry including, a method to allow the rapid input viahandwriting of the user answer. In some embodiments, for example, thiscan include providing a first region for receiving a first input,receiving the first input, determining when the first input iscompleted, providing a second region for receiving a second input,storing the first input, and determining if another input is beingreceived. In some embodiments, determining when the first input iscompleted can include, for example, receiving a signal indicative of thecompletion of the first input. The signal can be generated, for example,when the user selects the second region for providing input, when theuser actuates a key or button to generate a signal of the completion ofthe first input, after a specified amount of time has passed since theuser last took action resulting in the generation of a portion of thefirst input, and/or when the user takes any other action to generate asignal indicative of the completion of the first input.

With reference now to FIG. 1, a schematic illustration of one embodimentof a system for dynamic response entry 100 is shown. In someembodiments, for example, the system 100 can provide hardware, features,and/or software that can accelerate the rate with which a user can entera response. Advantageously, in some embodiments, these features canallow a test proctor to input answers received from a testee as quicklyas the testee is able to speak and/or indicate the answers.

The system 100 can include, a processor, one or several databases, oneor several user devices, one or several testing resources, and/or one orseveral user terminals. The system 100 can receive testing informationfrom the testing resource and/or from one of the databases, can providea task to the testee via one or several of the user devices, can receivean input and/or an answer from the testee via one or more of the userdevices, and can categorize the input and/or answer with the processorand/or one or more of the user devices.

The system 100 can include a processor 102. The processor 102 canprovide instructions to and receive information from the othercomponents of the system 100. The processor 102 can act according tostored instructions, which stored instructions can be located in memoryassociated with the processor 102 and/or in other components of thesystem 100. The processor 102 can comprise a microprocessor, such as amicroprocessor from Intel® or Advanced Micro Devices, Inc.®, or thelike.

The system 100 can include one or several databases 104. The one orseveral databases 104 can comprise stored data relevant to the functionsof the system 100. The one or several databases 104 include a userdatabase 104-A, a testing database 104-B, a battery database 104-C, alist database 104-D, a response database 104-E, score database 104-F,and/or a focus database 104-G.

In some embodiments, for example, the user database 104-A can compriseinformation relating to a user. This user can be, for example, aproctor, a testee, a supervisor, a recipient of the testing results,and/or any other desired individual or entity. In some embodiments, theuser database 104-A can include information relating to the user. Thisinformation can include, for example, a user profile. The user profilecan include information identifying the user. Including the user's name,age, birth date, race, gender, socioeconomic information, and/or anyother information useful in identifying the user and/or a characteristicof the user. In some embodiments, the user profile can includeexperiential information relating to the user and/or medical informationrelating to the user. In some embodiments, the experiential informationcan include information relating to the user's education and/oreducational level, relating to the user's past work experiences or toany other of the users important life experiences. In some embodiments,the medical information relating to the user can include details of theuser's medical history such as, for example, information relating to anymedical problems the user has had or that the user is prone to.

The user database 104-A can further include user login information. Thisinformation can include, for example, information identifying a usersuch as, for example, a username and password or a user identificationnumber. In some embodiments, for example, when a user desires to accessthe academic integrity system 100, the user can be prompted to enteridentification information which can include a username and password ora user identification number. After the user provides the identificationinformation, the academic integrity system 100 can verify theidentification information, and specifically, the processor 102 cancompare the user provided identification information to informationstored within the profile database 104-A to determine if the user is anauthorized user.

The testing database 104-B can include testing information. In someembodiments, for example, the testing information stored in the testingdatabase 104-B can include one or several tests. These one or severaltests can comprise one or several questions or tasks and data forevaluating and/or scoring the responses received to the one or severalquestions or tasks. In some embodiments, the testing information storedin the testing database 104-B can be specific to a user such as, forexample, the proctor, supervisor, or intended recipient of the testingresults. In some embodiments, the testing information stored in thetesting database 104-B is nonspecific to the testee user.

The battery database 104-C can include a subset of information from thetesting database 104-B that is associated with a testee. In someembodiments, for example, the battery database 104-C can include acompilation of one or several questions or tasks for providing to thetestee. The battery database 104-C can further include information forevaluating and/or scoring the responses received to the one or severalquestions or tasks associated with the testee in the battery database.

The list database 104-D can include information relating to a subset oftasks or questions associated with the testee in the battery database104-C. In some embodiments, for example, the task can include a list ofitems such as words. In performing the task, the proctor can temporarilyprovide these items to the testee, and can then ask the testee to recallas many of the items from the list as possible. In such an embodiment,the list database 104-D can include the list of items associated withthe task. In some further embodiments, some or all of the items from thelist can be associated with a group. This group can identify acommonality of the items. In one embodiment, for example, the list mayinclude “beet,” “carrot,” and “chard” which items may be associated withthe group “vegetable.” In some embodiments, this group can identifyitems on the list belonging to the group, as well as items that belongto the group and are not on the list. In some embodiments, the listdatabase 104-D can include information associated with an item thatidentifies one or several groups associated with the item, and any listassociated with the item.

The response database 104-E can comprise a data record of inputs and/ortestee answers provided in response to a question and/or task. In someembodiments, the response database 104-E can further include informationlinking each input and/or testee answer with the question and or taskthat it is associated with, with the battery and/or test with which itis associated, and/or the users, including, for example, the testeeand/or proctor, with which it is associated.

The score database 104-F can comprise data identifying a testee score ona question and/or task, on a collection of questions and/or tasks, onone or several tests and/or components thereof, or on the battery. Insome embodiments, the data in the score database 104-F can include dataassociating the score with the one or several questions and/or tasksthat prompted the input and/or testee answer leading to the score.

The focus database 104-F can comprise data identifying interrelationsbetween tasks in the testing database 104-B and/or battery database104-C, and/or interrelations between the one or several questions and/ortasks in the list database 104-D. In some embodiments, for example, thisdata identifying interrelations can include identifying the type ofinformation provided by one or several tasks and/or component thereof,including, for example, one or several questions and/or tasks.

In some embodiments, the information within the focus database 104-F canidentify one or several tests and/or test components that gatherinformation relating to an attribute of the testee. In some embodiments,one test may generally focus on the testee attribute, another test mayfocus on a specific aspect of the testee attribute, and an additionaltest may provide detail to a subcomponent of the specific aspect of thetestee attribute. Further, in some embodiments the focus database 104-Fcan include information identifying the different levels of test focus.Thus, the focus database 104-F can provide information that can be usedto narrowly target a testee attribute or aspect of the testee attributeby identifying the test that generates this information.

The system 100 can include one or several user devices 106. The one orseveral user devices 106 can be configured to allow a user tocommunicate with the other components of the system 100. In someembodiments, the user devices 106 can include a proctor device 106-A anda testee device 106-B. In some embodiments, the proctor device 106-A andthe testee device 106-B can be configured to communicate with each othervia other components of the system 100 and in some embodiments, theproctor device 106-A and the testee device 106-B can be configured todirectly communicate with each other. The details of the user devices106 will be discussed at greater length below.

The system 100 can include a testing resource 108. In some embodiments,for example, the testing resource 108 can comprise the source of teststhat are stored in the testing database 104-B. The testing resource 108can comprise, for example, a test vendor, a test developer, or a testencoder.

The system 100 can include a user terminal 110. The user terminal cancomprise features configured to allow a user to access the system 100.In some embodiments, for example, the user terminal 110 can beco-located in one or several of the user devices 106. In someembodiments, the user terminal can comprise a network interfaceconfigured to allow the user to interface with the other components ofthe system 100 and the user interface configured to allow the user tointerface with the user terminal 110. In some embodiments, the userterminal 110 can be used to generate a test battery stored in thebattery database 104-C from tests stored in the testing database 104-B.

The system 100 can include a network 112. The network 112 allowscommunication between the components of the system 100. The network 112can be, for example, a local area network (LAN), a wide area network(WAN), a wired network, wireless network, a telephone network such as,for example, a cellphone network, the Internet, the World Wide Web, orany other desired network. In some embodiments, the network 112 can useany desired communication and/or network protocols.

With reference now to FIG. 2, a block diagram of one embodiment of auser device 106 is shown. As discussed above, the user device 106 can beconfigured to provide information to and/or receive information fromother components of the system 100. The user device can access thesystem 100 through any desired means or technology, including, forexample, a webpage or a web portal. As depicted in FIG. 2, the userdevice 106 can include a network interface 200. The network interface200 allows the user device 106 to access the other components of thesystem 100, and specifically allows the user device 106 to access thenetwork 112 of the system 100. The network interface 200 can includefeatures configured to send and/or receive information, including, forexample, an antenna, a modem, a transmitter, receiver, or any otherfeature that can send and/or receive information. The network interface120 can communicate via telephone, cable, fiber-optic, or any otherwired communication network. In some embodiments, the network interface200 can communicate via cellular networks, WLAN networks, or any otherwireless network.

The user device 106 can include a user interface 202 that communicatesinformation to, and receives inputs from a user. The user interface 202can include a screen, a speaker, a monitor, a keyboard, a microphone, amouse, a touchpad, a keypad, and/or any other feature or features thatcan receive inputs from a user and provide information to a user.

The user device 106 can include a testing engine 204. The testing engine204 can be used to facilitate and/or manage the providing of a test to atestee. In some embodiments, for example, the testing engine 204 can beused by the proctor to provide and manage the testing provided to thetestee including, for example, selecting one or several tests forinclusion in the testing battery, editing the testing battery, and/orediting one or several of the tests included in the test battery.

The user device 106 can include a question/response engine 206. Thequestion/response engine 206 can facilitate providing a question and/ortask to the testee and providing a testee input and/or answer to thequestion and/or task. In some embodiments, for example, thequestion/response engine 206 can include features to facilitate theproctor in providing a question and/or task to the testee. In someembodiments in which the question and/or task includes temporarilyproviding a list of items to the testee, the question/response engine206 can include features to temporarily provide the list of items to thetestee. In some embodiments, for example, the list of items to betemporarily provided to the testee by temporarily displaying the itemsto the testee such as, for example, by the testee device 106-B, byprompting the proctor to read the items to the testee such as, forexample, by indicating that the proctor should read an item from thelist, indicating that the proctor should read all the items from thelist, and/or indicating a tempo and/or timing that the proctor canfollow in reading the items from the list, and/or by using speechsoftware and a feature of the user device 106 such as a speaker toaudibly communicate the items to the testee.

In some embodiments, the question/response engine 206 can includefeatures configured to facilitate the testee in providing an inputand/or answer to the question and/or task. In some embodiments, forexample, these features can include features configured to detect andrecord the testee's answers, to recognize the testee's answers such asby, for example, the use of speech recognition software, and/or toprovide the proctor with the ability to provide inputs to the system 100corresponding to the testee's answers. In some embodiments, for example,the proctor can provide inputs to the system 100 corresponding to thetestee's answers by selecting a word and/or item corresponding to thetestee answer, by typing the testee answer, by handwriting the testeeanswer, or by verbally repeating the testee answer.

The user device 106 includes a scoring engine 208. The scoring engine208 can be configured to evaluating score of the one or several testeeinputs and/or answers. In some embodiments, for example, scoring engine208 can be configured to retrieve scoring information from the scoringdatabase 104-F and/or from the testing database 104-B, and to generate ascore for the one or several inputs and/or answers.

The user device 106 includes a focus engine 210. The focus engine 210can be configured to retrieve the score from the scoring engine 208and/or from the scoring database 104-F, and to receive focus informationfrom the focus database 104-G. In some embodiments, for example, thefocus engine 210 can use the score and the information received from thefocus database 104-G to determine further tests that can be provided tothe testee to further identify and and/or evaluate a testee attribute.In some embodiments, for example, this testee attribute can be selectedby the intended recipient of the testing results, and in someembodiments, this testee attribute can be selected based on thecomparison of the testee achieved score to a threshold. Thus, in someembodiments in which the testee score is less than, the same as, or morethan the threshold value, the focus engine 210 can recommend additionaltesting based on the testee performance.

With reference now to FIG. 3, a schematic illustration of one embodimentof a user interface 202 performing dynamic response entry is shown. Insome embodiments, for example, the user interface 202 can be provided onone or more of the user devices 106 and/or on the user terminal 110. Insome embodiments, for example, the user interface 202 can includefeatures that can be configured to facilitate the proctor in dynamicresponse entry. The user interface 202 can operate according to thesoftware and components of the one or several user devices 106.

The user interface 202 can comprise an answer list. The answer list 302can comprise an area in which the testee provided one or several inputsand/or answers can be displayed. The answer list 302 can include an itemdisplay 304 in which a testee provided item can be displayed, and astatus display 306 in which the status of the testee provided item canbe displayed. In some embodiments, for example, these answers can bedisplayed in a plurality of item displays 304 in a list format and canbe chronologically sorted according to when the input and or answer wasreceived. In some embodiments, for example, a status associated with theone or more inputs and/or answers can be displayed in the status display306 of answer list 302.

The user interface 202 can include a button region 308. In someembodiments, for example, the button region 308 can include a pluralityof buttons. As depicted in FIG. 3, the button region 308 includes aninput button 310. The input button 310 can allow a user to select anitem from the list of items by operating the input button 310. In oneembodiment in which the user devices 106 include, for example, a touchscreen, the input button 310 can be operated by touching the portion ofthe touchscreen overlaying the displayed input button 310. As furtherseen in FIG. 3, the button region 308 can include a status button 312.In some embodiments, for example, the status button 312 can allow a userto select a status for an input item. In some embodiments, for example,a status can be assigned to an input item including, for example,whether the input item is a repeat input item in that it has beenprovided multiple times by the testee, whether the item is an intrusionin that it is not on the list, whether the item is non-category in thatit does not belong to a group associated with the list, and/or whetherthe item is a synonym for an already provided and/or listed item. Insome embodiments, the status button 312 can be operated at the time thatan input is provided and can be used to assign a status to the input. Insome embodiments, the status button 312 can be used to edit and/orupdate the status of an item that has already been input and providedwith status.

The user interface 202 can include a manual input region 314. In someembodiments, for example, the manual input region 314 can includefeatures configured to allow a user such as, for example, the proctor toinput a testee answer into the user device. This manual input region canbe configured to allow a user such as a proctor to input a testee answervia, for example, a microphone, a keyboard, and/or handwriting. In someembodiments, the manual input region 314 can include a button 316configured to indicate when an input of a testee answer is complete, anda display box 318 for displaying of the user provided testee answer. Inthe specific embodiment depicted in FIG. 3, the manual input region 314includes a first display box 318-A and a second display box 318-B. Insome embodiments, the first and second display boxes 318-A, 318-B can besimultaneously provided to the user, and in some embodiments, the firstand second display boxes 318-A, 318-B can be serially provided to theuser. Thus, in some embodiments, the first and second display boxes318-A, 318-B can be both provided to the user in the manual input region314 at the same time, and in some embodiments, the first and seconddisplay boxes 318-A, 318-B can be serially provided to the user so thata new display box is provided when the input in the previous display boxis completed.

As further depicted in FIG. 3, the first display box 318-A is associatedwith a first button 316-A and the second display box 318-B is associatedwith a second button 316-B. The details of the use of the display boxes318-A, 318-B and the buttons 316-A, 316-B will be provided in below.

With reference now to FIG. 4, a flowchart illustrating one embodiment ofa process 400 for dynamic response entry is shown. In some embodiments,for example, the process 400 can be performed by the system 100 and/orone or several components of the system 100.

The process 400 begins at block 402 wherein a task and/or question isreceived. In some embodiments, for example, receiving the task and/orquestion can include querying at least one of the databases 104 forinformation relating to the task and/or question. In some embodiments,for example, receiving the task and/or question can include the processof evaluating the test located in the testing database 104-B, compilingthe testing battery with tasks from the testing database 104-B, storingthe testing battery within the battery database 104-C, identifying atesting battery stored in the battery database 104-C and associated withthe testee, retrieving the identified testing battery from the database104-C, identifying a task and/or question associated with the testingbattery stored in the battery database 104-C and any list associatedwith the task and/or question, and retrieving the identified task and/orquestion and any list associated with the task and/or question from thelist database 104-D.

After the task has been received, the process 400 proceeds to block 404wherein the task and/or question is provided. In some embodiments, forexample, the task and/or question can be provided to a user such as theproctor and/or the testee. The task and/or question can, for example, beprovided to the user by at least one of the user devices 106, andspecifically by, for example, the user interface 202 of the user device106. In some embodiments, the task and/or question can comprise arequest for a user such as the testee to take some action, which caninclude, for example, after being temporarily provided with a list ofitems such as, for example, names of objects, recalling as many of theitems as possible.

After the task has been provided, the process 400 proceeds to block 406wherein the input and/or answer is received. The input and/or answer canbe received in response to the provided task. In some embodiments, forexample, the input can be received by at least one of the user devices106, and in some embodiments, the input can be received from the proctordevice 106-A. The received input and or answer can be stored in, forexample, one of the databases 104 such as the response database 104-E.In some embodiments, for example, the receipt of the input and/or answercan further comprise determining the format and/or data type of theinput and/or answer and determining whether the format and/or data typeof the input and/or answer is a desired format and/or data type such as,for example, the format and/or data type matching the format and/or datatype of the list data retrieved from the list database 104-D. If theformat and/or data type of the input does not match the desired formatand/or data type, receiving the input and/or answer can further compriseconverting the format and/or data type of the input to match and/orcorrespond to the format and/or data type of the list data.

After the input has been received, the process 400 proceeds to block 408wherein a group is identified. In some embodiments, for example, thegroup can be identified by retrieving this data from the list database104-D including group data. This group data can identify a groupassociated with one or several of the items in the list.

After the group has been identified, the process proceeds to decisionstate 410 wherein it is determined if the received input is in the list.In some embodiments, this determination can include comparing the inputto items on the list and determining whether the input is on the list.In some embodiments, for example, this can include identifying,according to a Boolean function, inputs identified in the list data. Ifit is determined that the input is on the list, then the process 400proceeds to block 412 wherein an indicator of the input's inclusion onthe list is provided. In some embodiments, for example, the providing ofthe indicator of the input's inclusion on the list can include, forexample, the addition of an indicator to the response database 104-E. Insome embodiments, for example, the indicator can comprise a firstBoolean value corresponding to a “true” condition. Returning again todecision state 410, if it is determined that the input is not on thelist, then the process 400 proceeds to block 414, wherein an indicatorof the non-inclusion of the input on the list is provided. In someembodiments, for example, the providing of the indicator of thenon-inclusion of the input on the list can include, for example, theaddition of an indicator to the response database 104-E. In someembodiments, for example, the indicator can comprise a second Booleanvalue corresponding to a “false” condition.

After the indicator of the inclusion and/or non-inclusion of the inputin the list is added, the process 400 proceeds to decision state 416wherein it is determined if the input is in a group. In someembodiments, this determination can include comparing the groupassociated with the input groups associated with the items on the listand determining whether the group associated with the input matches oneof the groups associated with the items on the list. In someembodiments, for example, this can include identifying, according to aBoolean function, whether the group associated with the input matches atleast one group associated with at least one of the items on the list.If it is determined that the group associated with the input matches oneof the groups associated with at least one of the items on the list,then the process 400 proceeds to block 418 wherein an indicator of thematching group is provided. In some embodiments, for example, theproviding of the indicator of the matching group can include, forexample, the addition of an indicator to the response database 104-E. Insome embodiments, for example, the indicator can comprise a firstBoolean value corresponding to a “true” condition.

Returning again to decision state 416, if it is determined that thegroup associated with the input does not match a group associated withat least one of the items on the list, then the process 400 proceeds toblock 420, wherein an indicator of the non-matching group is provided.In some embodiments, for example, the providing of the indicator of thenon-matching group can include, for example, the addition of anindicator to the response database 104-E. In some embodiments, forexample, the indicator can comprise a second Boolean value correspondingto a “false” condition.

After the indicator of the group matching and/or non-matching is added,the process 400 proceeds to block 422 wherein the received input isevaluated. In some embodiments, for example, the evaluation of thereceived input can be performed by, for example, a component of thesystem 100 such as, for example, the processor 102, at least one of theuser devices 106, and/or the user terminal 110. In some embodiments, forexample, the evaluation can include evaluating the received input forrepeat and/or synonym status.

After the received input is evaluated, the process 400 proceeds todecision state 424 wherein it is determined if the input is a repeatinput. In some embodiments, this determination can include comparingresponse data from the response database 104-E with the received inputto determine if the received input matches a previously received input.In some embodiments, for example, this determination can includeidentifying any inputs provided in response to a question and/or taskand comparing the received input to the identified inputs. In someembodiments, for example, this can include determining, according to aBoolean function, whether the received input is a repeat of one of theidentified inputs.

If it is determined that the received input is a repeat of one of theidentified inputs, then the process 400 proceeds to block 426 wherein anindicator of the repeat input is provided. In some embodiments, forexample, the providing of the indicator of the repeat input can include,for example, the addition of an indicator to the response database104-E. In some embodiments, for example, the indicator can comprise afirst Boolean value corresponding to a “true” condition.

Returning again to decision state 424, if it is determined that thereceived input is not a repeat of one of the identified inputs, then theprocess 400 proceeds to block 428, wherein an indicator of the repeatinput is provided. In some embodiments, for example, the providing ofthe indicator of the non-repeat input can include, for example, theaddition of an indicator to the response database 104-E. In someembodiments, for example, the indicator can comprise a second Booleanvalue corresponding to a “false” condition.

After the indicator of the repeat and/or non-repeat input is added, theprocess 400 proceeds to block 430 wherein the input and status isdisplayed. In some embodiments, for example, the input and status can bedisplayed in the answer list 302 of the user interface 202. In someembodiments, for example, this display in the answer list 302 of theuser interface 202 can be performed on at least one of the user devices106, and in some embodiments on the proctor device 106-A.

With reference now to FIG. 5, a flowchart illustrating one embodiment ofa process 500 for receiving an input is provided. In some embodimentsthe process 500 can be performed as a sub-process within block 406 ofFIG. 4. The process 500 can be performed by the system 100 and/or acomponent of the system 100.

The process 500 begins at block 502 wherein an input region is provided.In some embodiments, for example, the input region can comprise aportion of the user interface 202 such as, for example, the buttonregion 308 and/or the manual input region 314 of the user interface 202displayed in FIG. 3. In one embodiment, the input region comprises thedisplay box 318, and can comprise, for example, the first display box318-A of the manual input region 314. In some embodiments, the providingof the input region can include providing the user interface 202 asshown in FIG. 3, or any of the portions thereof including, for example,the button region 308.

After the input has been provided, the process 500 proceeds to block 504wherein an input is received. In some embodiments, for example, theinput can be received by the interaction of the user such as the proctorwith the user interface 202. In one particular embodiment, for example,the input can be received by the interaction of the user such as theproctor with the manual input region 314 of the user interface. Thisinteraction can include, for example, the proctor handwriting the inputinto the first display box 318-A.

After the input is received, the process 500 proceeds to block 506wherein an indication of the completion of the input is received. Insome embodiments, for example, the indication the completion of theinput can be received by the system 100 and/or a component of the system100 such as, for example, at least one of the user devices 106. In someembodiments, for example, the indication of the completion of the inputcan comprise the operation the button 316 by the user such as theproctor, a lapse of the designated period of time during which no userinput is received, and/or the receipt of an additional input. In someembodiments, for example, the receipt of an additional input can bedetermined by identifying the start of providing an additional inputinto, for example, an additional display box 318 such as, the seconddisplay box 318-B of the manual input region 314. In some embodiments,an input status can be determined for the received input according toprocess 400 of FIG. 4.

After the indication of the completion of the input is received, theprocess 500 proceeds to block 508 wherein an additional input region isprovided for an additional input. In some embodiments, for example, theadditional input region can comprise a portion of the user interface 202such as, for example, the button region 308 and/or the manual inputregion 314 of the user interface 202 displayed in FIG. 3. In oneembodiment, the input region comprises the display box 318, and cancomprise, for example, the second display box 318-B of the manual inputregion 314. In some embodiments, in which the first and second displayboxes 318-A, 318-B are simultaneously provided, the step of block 508can be performed simultaneously with the step of block 502, and in someembodiments in which the first and second display boxes 318-A, 318-B areserially provided, the step of block 508 can be performed in response tothe receipt of an indication of the completion of an input in a displaybox 318.

After the additional input region is provided for the additional input,the process 500 proceeds to block 510 wherein the received input isstored. In some embodiments, for example, the received input can bestored in a component of the system 100 such as, for example, one of thedatabases 104, and specifically in the response database 104-E.

After the received input is stored, the process 500 proceeds to decisionstate 512 wherein it is determined if an additional input is beingprovided. In some embodiments, for example, this determination caninclude determining whether an additional input is being provided in,for example, the additional display box 318 such as the second displaybox 318-B of the manual input region 314. If it is determined that anadditional input is being provided, the process 500 returns to block504. If it is determined that no additional input is being provided, theprocess 500 can terminate and or proceed to block 408 of FIG. 4.

With reference now to FIG. 6, an exemplary environment with whichembodiments may be implemented is shown with a computer system 600 thatcan be used by a user 604 as all or a component of the system fordynamic response entry 100. The computer system 600 can include acomputer 602, keyboard 622, a network router 612, a printer 608, and amonitor 606. The monitor 606, processor 602 and keyboard 622 are part ofa computer system 626, which can be a laptop computer, desktop computer,handheld computer, mainframe computer, etc. The monitor 606 can be aCRT, flat screen, etc.

A user 604 can input commands into the computer 602 using various inputdevices, such as a mouse, keyboard 622, track ball, touch screen, etc.If the computer system 600 comprises a mainframe, a designer 604 canaccess the computer 602 using, for example, a terminal or terminalinterface. Additionally, the computer system 626 may be connected to aprinter 608 and a server 610 using a network router 612, which mayconnect to the Internet 618 or a WAN.

The server 610 may, for example, be used to store additional softwareprograms and data. In one embodiment, software implementing the systemsand methods described herein can be stored on a storage medium in theserver 610. Thus, the software can be run from the storage medium in theserver 610. In another embodiment, software implementing the systems andmethods described herein can be stored on a storage medium in thecomputer 602. Thus, the software can be run from the storage medium inthe computer system 626. Therefore, in this embodiment, the software canbe used whether or not computer 602 is connected to network router 612.Printer 608 may be connected directly to computer 602, in which case,the computer system 626 can print whether or not it is connected tonetwork router 612.

With reference to FIG. 7, an embodiment of a special-purpose computersystem 704 is shown. The above methods may be implemented bycomputer-program products that direct a computer system to perform theactions of the above-described methods and components. Each suchcomputer-program product may comprise sets of instructions (codes)embodied on a computer-readable medium that directs the processor of acomputer system to perform corresponding actions. The instructions maybe configured to run in sequential order, or in parallel (such as underdifferent processing threads), or in a combination thereof. Afterloading the computer-program products on a general purpose computersystem 626, it is transformed into the special-purpose computer system704.

Special-purpose computer system 704 comprises a computer 602, a monitor606 coupled to computer 602, one or more additional user output devices730 (optional) coupled to computer 602, one or more user input devices740 (e.g., keyboard, mouse, track ball, touch screen) coupled tocomputer 602, an optional communications interface 750 coupled tocomputer 602, a computer-program product 705 stored in a tangiblecomputer-readable memory in computer 602. Computer-program product 705directs system 704 to perform the above-described methods. Computer 602may include one or more processors 760 that communicate with a number ofperipheral devices via a bus subsystem 790. These peripheral devices mayinclude user output device(s) 730, user input device(s) 740,communications interface 750, and a storage subsystem, such as randomaccess memory (RAM) 770 and non-volatile storage drive 780 (e.g., diskdrive, optical drive, solid state drive), which are forms of tangiblecomputer-readable memory.

Computer-program product 705 may be stored in non-volatile storage drive780 or another computer-readable medium accessible to computer 602 andloaded into memory 770. Each processor 760 may comprise amicroprocessor, such as a microprocessor from Intel® or Advanced MicroDevices, Inc.®, or the like. To support computer-program product 705,the computer 602 runs an operating system that handles thecommunications of product 705 with the above-noted components, as wellas the communications between the above-noted components in support ofthe computer-program product 705. Exemplary operating systems includeWindows® or the like from Microsoft® Corporation, Solaris® from Oracle®,LINUX, UNIX, and the like.

User input devices 740 include all possible types of devices andmechanisms to input information to computer system 602. These mayinclude a keyboard, a keypad, a mouse, a scanner, a digital drawing pad,a touch screen incorporated into the display, audio input devices suchas voice recognition systems, microphones, and other types of inputdevices. In various embodiments, user input devices 740 are typicallyembodied as a computer mouse, a trackball, a track pad, a joystick,wireless remote, a drawing tablet, a voice command system. User inputdevices 740 typically allow a user to select objects, icons, text andthe like that appear on the monitor 606 via a command such as a click ofa button or the like. User output devices 730 include all possible typesof devices and mechanisms to output information from computer 602. Thesemay include a display (e.g., monitor 606), printers, non-visual displayssuch as audio output devices, etc.

Communications interface 750 provides an interface to othercommunication networks 795 and devices and may serve as an interface toreceive data from and transmit data to other systems, WANs and/or theInternet 618. Embodiments of communications interface 750 typicallyinclude an Ethernet card, a modem (telephone, satellite, cable, ISDN), a(asynchronous) digital subscriber line (DSL) unit, a FireWire®interface, a USB® interface, a wireless network adapter, and the like.For example, communications interface 750 may be coupled to a computernetwork, to a FireWire® bus, or the like. In other embodiments,communications interface 750 may be physically integrated on themotherboard of computer 602, and/or may be a software program, or thelike.

RAM 770 and non-volatile storage drive 780 are examples of tangiblecomputer-readable media configured to store data such ascomputer-program product embodiments of the present invention, includingexecutable computer code, human-readable code, or the like. Other typesof tangible computer-readable media include floppy disks, removable harddisks, optical storage media such as CD-ROMs, DVDs, bar codes,semiconductor memories such as flash memories, read-only-memories(ROMs), battery-backed volatile memories, networked storage devices, andthe like. RAM 770 and non-volatile storage drive 780 may be configuredto store the basic programming and data constructs that provide thefunctionality of various embodiments of the present invention, asdescribed above.

Software instruction sets that provide the functionality of the presentinvention may be stored in RAM 770 and non-volatile storage drive 780.These instruction sets or code may be executed by the processor(s) 760.RAM 770 and non-volatile storage drive 780 may also provide a repositoryto store data and data structures used in accordance with the presentinvention. RAM 770 and non-volatile storage drive 780 may include anumber of memories including a main random access memory (RAM) to storeof instructions and data during program execution and a read-only memory(ROM) in which fixed instructions are stored. RAM 770 and non-volatilestorage drive 780 may include a file storage subsystem providingpersistent (non-volatile) storage of program and/or data files. RAM 770and non-volatile storage drive 780 may also include removable storagesystems, such as removable flash memory.

Bus subsystem 790 provides a mechanism to allow the various componentsand subsystems of computer 602 communicate with each other as intended.Although bus subsystem 790 is shown schematically as a single bus,alternative embodiments of the bus subsystem may utilize multiple bussesor communication paths within the computer 602.

A number of variations and modifications of the disclosed embodimentscan also be used. Specific details are given in the above description toprovide a thorough understanding of the embodiments. However, it isunderstood that the embodiments may be practiced without these specificdetails. For example, well-known circuits, processes, algorithms,structures, and techniques may be shown without unnecessary detail inorder to avoid obscuring the embodiments.

Implementation of the techniques, blocks, steps and means describedabove may be done in various ways. For example, these techniques,blocks, steps and means may be implemented in hardware, software, or acombination thereof. For a hardware implementation, the processing unitsmay be implemented within one or more application specific integratedcircuits (ASICs), digital signal processors (DSPs), digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, other electronic units designed toperform the functions described above, and/or a combination thereof.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a swim diagram, a dataflow diagram, a structure diagram, or a block diagram. Although adepiction may describe the operations as a sequential process, many ofthe operations can be performed in parallel or concurrently. Inaddition, the order of the operations may be re-arranged. A process isterminated when its operations are completed, but could have additionalsteps not included in the figure. A process may correspond to a method,a function, a procedure, a subroutine, a subprogram, etc. When a processcorresponds to a function, its termination corresponds to a return ofthe function to the calling function or the main function.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages, and/or any combination thereof. When implementedin software, firmware, middleware, scripting language, and/or microcode,the program code or code segments to perform the necessary tasks may bestored in a machine readable medium such as a storage medium. A codesegment or machine-executable instruction may represent a procedure, afunction, a subprogram, a program, a routine, a subroutine, a module, asoftware package, a script, a class, or any combination of instructions,data structures, and/or program statements. A code segment may becoupled to another code segment or a hardware circuit by passing and/orreceiving information, data, arguments, parameters, and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

For a firmware and/or software implementation, the methodologies may beimplemented with modules (e.g., procedures, functions, and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing themethodologies described herein. For example, software codes may bestored in a memory. Memory may be implemented within the processor orexternal to the processor. As used herein the term “memory” refers toany type of long term, short term, volatile, nonvolatile, or otherstorage medium and is not to be limited to any particular type of memoryor number of memories, or type of media upon which memory is stored.

Moreover, as disclosed herein, the term “storage medium” may representone or more memories for storing data, including read only memory (ROM),random access memory (RAM), magnetic RAM, core memory, magnetic diskstorage mediums, optical storage mediums, flash memory devices and/orother machine readable mediums for storing information. The term“machine-readable medium” includes, but is not limited to portable orfixed storage devices, optical storage devices, and/or various otherstorage mediums capable of storing that contain or carry instruction(s)and/or data.

While the principles of the disclosure have been described above inconnection with specific apparatuses and methods, it is to be clearlyunderstood that this description is made only by way of example and notas limitation on the scope of the disclosure.

What is claimed is:
 1. A system, comprising: a battery databaseincluding a plurality of tasks, wherein each task in the plurality oftasks is associated with task contents; and a proctor device, theproctor device including: a display device, and a processor, theprocessor being configured to perform the steps of: retrieving a taskfrom the battery database; causing a user device to display contents ofa task via a user interface of the user device and requesting a user torespond to the task, wherein the user interface of the user deviceincludes an input area configure to receive an input from the userresponsive to the contents of the task; determining an evaluationinterface configured to enable an evaluation of the input from the user;displaying, on the display device of the proctor device, the evaluationinterface; receiving the input from the user from the user device;displaying an indication of the input from the user on the evaluationinterface; and storing the input.
 2. The system of claim 1, wherein theprocessor is configured to perform the steps of receiving a status ofthe input, wherein the status of the input indicates at least one ofwhether the input is included in a list or group and whether the inputis a repeat input.
 3. The system of claim 2, wherein the processor isconfigured to perform the steps of determining, by the proctor device,the status of the input, wherein determining the status of the inputcomprises retrieving categorization data from a database and comparingthe categorization data to the input.
 4. The system of claim 3, whereinthe categorization data comprises one of list data, wherein the listdata comprises data indicating a plurality of items, group data, whereinthe group data identifies a subset of the plurality of items in the listdata, and input data, wherein the input data identifies a previouslyreceived input.
 5. The system of claim 1, wherein the processor isconfigured to perform the steps of causing the user interface to displaya button region, wherein the button region comprises a plurality ofinput buttons.
 6. The system of claim 5, wherein each of the pluralityof input buttons is associated with an item identified in list data,wherein the list data comprises data indicating a plurality of items. 7.The system of claim 1, wherein the user interface is configured tocapture an audio recording of the user of the user device, wherein theaudio recording is response to the content of the task.
 8. A device,comprising: a display device; and a processor, the processor beingconfigured to perform the steps of: causing a user device to displaycontents of a task via a user interface of the user device andrequesting a user to respond to the task, wherein the user interface ofthe user device includes an input area configure to receive an inputfrom the user responsive to the contents of the task; determining anevaluation interface configured to enable an evaluation of the inputfrom the user; displaying, on the display device, the evaluationinterface; receiving the input from the user from the user device;displaying an indication of the input from the user on the evaluationinterface; and storing the input.
 9. The device of claim 8, wherein theprocessor is configured to perform the steps of receiving a status ofthe input, wherein the status of the input indicates at least one ofwhether the input is included in a list or group and whether the inputis a repeat input.
 10. The device of claim 9, wherein the processor isconfigured to perform the steps of determining the status of the input,wherein determining the status of the input comprises retrievingcategorization data from a database and comparing the categorizationdata to the input.
 11. The device of claim 10, wherein thecategorization data comprises one of list data, wherein the list datacomprises data indicating a plurality of items, group data, wherein thegroup data identifies a subset of the plurality of items in the listdata, and input data, wherein the input data identifies a previouslyreceived input.
 12. The device of claim 8, wherein the processor isconfigured to perform the steps of causing the user interface to displaya button region, wherein the button region comprises a plurality ofinput buttons.
 13. The device of claim 12, wherein each of the pluralityof input buttons is associated with an item identified in list data,wherein the list data comprises data indicating a plurality of items.14. The device of claim 8, wherein the user interface is configured tocapture an audio recording of the user of the user device, wherein theaudio recording is response to the content of the task.
 15. A method,comprising: causing, by a proctor device, a user device to displaycontents of a task via a user interface of the user device andrequesting a user to respond to the task, wherein the user interface ofthe user device includes an input area configure to receive an inputfrom the user responsive to the contents of the task; determining, bythe proctor device, an evaluation interface configured to enable anevaluation of the input from the user; displaying, on a user interfaceof the proctor device, the evaluation interface; receiving, by theproctor device, the input from the user from the user device;displaying, by the proctor device, an indication of the input from theuser on the evaluation interface; and storing, by the proctor device,the input.
 16. The method of claim 15, further comprising receiving astatus of the input, wherein the status of the input indicates at leastone of whether the input is included in a list or group and whether theinput is a repeat input.
 17. The method of claim 16, further comprisingdetermining, by the proctor device, the status of the input, whereindetermining the status of the input comprises retrieving categorizationdata from a database and comparing the categorization data to the input.18. The method of claim 17, wherein the categorization data comprisesone of list data, wherein the list data comprises data indicating aplurality of items, group data, wherein the group data identifies asubset of the plurality of items in the list data, and input data,wherein the input data identifies a previously received input.
 19. Themethod of claim 15, further comprising providing a button region in theuser interface, wherein the button region comprises a plurality of inputbuttons.
 20. The method of claim 15, wherein the user interface isconfigured to capture an audio recording of the user of the user device,wherein the audio recording is response to the content of the task.